Although large scale testing, review, and auditing processes often provide valuable business and organization data, these processes may take substantial amounts of time and consume valuable organization resources. For example, performing an auditing process on a large set of business documents may require employees to individually review, classify, score, and record the result for each separate document. A typical large scale review may include thousands or even millions of documents, each of which may be complex and time-consuming to review and classify. For instance, assume a business wanted to audit a set of service contracts to confirm that its quality control processes (e.g., automated tools, employee training, internal reviews, etc.) were working adequately to ensure that the contracts were being structured properly, used the correct language, and were being filled out correctly by employees and clients. However, while preparing for the audit, the business determined that 20,000 contacts were available for review, and each contract would take a human auditor approximately 1-2 hours to review. Thus, a comprehensive audit of its service contracts would cost the business approximately 30 years in employee work time. As another example, an internal customer service review may involve analyzing written transcripts of customer-agent telephone conversations to identify certain characteristics and criteria of the customer interactions. As in the previous example, if the organization attempted a comprehensive review of every transcript, the cost in employee time and other organization resources may be very large. As yet another example, an organization may want to review each of a large number of customer survey responses to find specific responses within the customer feedback. As these examples illustrate, comprehensive testing, auditing, and reviewing processes for large scale projects is often a costly proposition.
Sampling and statistical analysis provide businesses a way to perform testing processes on large amounts of data without having to test each item in the population. Sampling is a well known technique in the field of statistical analysis by which a sample set of individual items within the population are tested for the purpose of drawing statistical inferences about the population as a whole. Thus, by reviewing a relatively small number of items (e.g., contracts, transcripts, etc.) selected at random from the population, it may be possible to reach valid statistical conclusions about the overall population with a high degree of certainty. However, in many cases, the cost of reviewing even a small percentage of the overall population of items may be significant. In the above example of service contract auditing, if only 5% of the contracts were reviewed, the overall cost to the company would still exceed one year in employee work time. Furthermore, the precision of statistical inferences generally depend on the size of the sample sets reviewed. Thus, organizations that are compelled to select larger sample sets to increase their data precision will further increase their testing costs. Finally, although some sample sets do yield valid statistical inferences about the population as a whole, other sample sets might only yield inconclusive results and will therefore require additional testing before an adequate inference can be drawn.